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Abstract 

The  American  Osteopathic  Association’s  Clinical  Assessment  Program  (AOA- 
CAP)  provides  a  mechanism  for  osteopathic  residency  programs  to  measure  and 
improve  their  quality  of  patient  care.  In  CAP,  current  clinical  practices  are 
measured,  and  then  compared,  to  evidence-based  practice  guidelines  representing 
state-of-the-art  professional  standards,  such  as  The  National  Committee  on 
Quality  Assurance  (NCQA)  Health  Plan  Employer  Data  and  Infonnation  Set 
(HEDIS),  Healthy  People  2010  targets,  and  recommendations  from  the  American 
Diabetes  Association.  In  this  program,  data  abstracted  directly  from  patients’ 
medical  records  are  analyzed  to  determine  the  residents’  performance  as  well  as 
the  impact  and  effectiveness  of  residency  program  treatment  protocols  in  meeting 
standards  of  practice  for  certain  clinical  categories  of  patients.  These  categories 
are  women’s  health,  childhood  immunizations,  adult  immunizations,  low  back 
pain,  hypertension,  coronary  artery  disease,  and  diabetes  mellitus.  All  osteopathic 
family  practice  programs  are  required  to  participate  in  the  AOA-CAP  program  as 
part  of  the  accreditation  process.  This  information  is  then  used,  when  necessary, 
to  modify  residents’  clinical  behavior  and  teaching  programs.  Improvement  is 
documented  through  re-evaluation.  Our  results  thus  far  support  the  suggestion  of 
NCQA  that  significant  “quality  gaps”  exist.  By  reducing  these  quality  gaps, 
which,  according  to  NCQA,  result  in  more  than  57,000  U.S.  deaths  annually, 
patient  safety  is  certainly  being  advanced.  Through  a  platform  of  process  sharing, 
the  AOA-CAP  provides  specific  infonnation  to  residency  program  directors  about 
how  to  improve  the  quality  of  patient  care. 


Introduction 

Health  care  providers  in  America  have  come  under  increasing  pressure  over 
the  past  decade  to  develop  systems  that  assure  the  provision  of  safe  patient  care. 
The  Institute  of  Medicine  (IOM)  crystallized  concerns  regarding  patient  safety 
with  the  1999  report  To  Err  Is  Human,  stating  that  between  48,000  and  98,000 
deaths  annually  are  due  to  medical  error.1  These  concerns  are  increasing  as  health 
care  becomes  more  costly  to  Americans;  in  1999  Americans  invested  1.1  trillion 
dollars  in  health  care — 13.5  percent  of  the  Nation’s  gross  domestic  product 
(GDP).  This  figure  is  expected  to  rise  to  over  2  trillion  dollars,  or  16  percent  of 
GDP  by  2007. 2 

Although  the  IOM  report  focused  on  inpatient  errors,  it  sparked  a  great 
amount  of  interest  in  medical  errors  in  all  types  of  health  care  delivery  and  helped 
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set  the  taxonomy  for  patient  safety.  Safe  patient  care  has  been  characterized  as 
having  three  agendas:  identifying  what  works  (evidence-based),  ensuring  that  the 
patient  receives  what  works  (delivery  consistent  with  the  evidence),  and 
delivering  it  without  errors.3  The  Agency  for  Healthcare  Research  and  Quality 
(AHRQ)  has  defined  a  number  of  practices  for  patient  safety  through  the 
publication  of  Evidence  Report  43. 4  This  report  ranked  practices  to  improve 
patient  safety  based  on  the  strength  of  evidence  behind  the  practice  and  began  to 
shape  the  agenda  for  patient  safety. 

To  Err  Is  Human  focused  primarily  on  errors  of  commission,  such  as  wrong- 
site  surgery.  The  AHRQ  report  expanded  the  patient  safety  agenda  to  include 
errors  of  omission,  such  as  not  providing  perioperative  beta-blockade  to  patients 
at  risk  for  cardiac  events,  thus  increasing  the  patient’s  risk  of  an  adverse  event  and 
reducing  the  safety  of  their  care. 

When  evaluating  patient  safety  in  the  outpatient  setting,  especially  when 
focusing  on  ambulatory  care,  the  largest  opportunities  for  improvement  exist  in 
delivering  consistent  evidenced-based  care,  thereby  reducing  the  number  of  errors 
of  omission.  The  National  Committee  for  Quality  Assurance  (NCQA)  has 
suggested  that  42,000  to  79,000  deaths  annually  would  be  prevented  if  all 
Americans  were  receiving  the  same  consistent  care,  in  10  categories,  as  those  who 
are  under  treated  by  the  country’s  top  10  health  plans.3  The  categories  are  beta- 
blocker  treatment,  breast  cancer  screening,  cervical  cancer  screening,  cholesterol 
management,  colorectal  cancer  screening,  controlling  high  blood  pressure, 
controlling  diabetes,  flu  shots  for  adults  over  65,  prenatal  care,  and  smoking 
cessation. 


The  American  Osteopathic  Association  Clinical 
Assessment  Program  (AOA-CAP) 

The  American  Osteopathic  Association  (AO A)  is  a  professional  membership 
organization  representing  the  approximately  52,000  osteopathic  physicians  (DOs) 
practicing  in  the  United  States  and  abroad.  The  AOA  works  to  promote  timely 
public  health  issues,  encourages  scientific  research,  and  serves  as  the  primary 
certifying  body  for  all  20  osteopathic  medical  schools  and  hundreds  of  health  care 
facilities  across  the  nation.  The  AOA  has  been  tracking  the  concerns  raised  by  the 
IOM,  AHRQ,  and  others  regarding  health  care  delivery  in  America,  and  has, 
through  various  committees  and  taskforces,  developed  its  patient  safety  agenda  as 
a  response  to  these  concerns.  With  a  large  percentage  of  its  members  involved  in 
providing  primary  care  to  patients,  the  AOA’s  efforts  have  focused  on  methods  to 
increase  the  safety  of  care  delivered  in  the  ambulatory  environment.  The  AOA 
believes  the  largest  opportunity  for  improvement  in  the  ambulatory  setting  is  in 
assuring  evidenced-based  medicine  is  delivered  consistently  and  has  explored 
methods  that  could  be  used  to  achieve  this  goal.  Subsequently,  the  AOA  made  the 
commitment  to  develop  a  voluntary  registry,  called  the  Clinical  Assessment 
Program  (AOA-CAP),  to  serve  as  a  quality  improvement  tool  for  physicians  in 
training  to  evaluate  the  safety  of  their  patient  care  in  the  ambulatory  setting.  This 
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type  of  program  introduces  physicians  to  active  participation  in  quality 
improvement  activities  while  still  in  their  graduate  medical  education  training. 

The  AOA  developed  several  guiding  principals  for  the  program,  including: 

•  The  use  of  information  technology  to  drive  the  program.  This  resulted 
in  the  development  of  an  AOA-supported,  Web-based  architecture 
providing  key  components  to  participating  programs  and  physicians. 

•  The  standardization  of  the  AOA’s  measures  to  national  efforts, 
including  the  NCQA  Health  Employer  Data  and  Infonnation  Set 
(HEDIS)  measures,  Center  for  Medicare  and  Medicaid  Services 
(CMS)  Doctors  Office  Quality  Initiative  (DOQ),  and  Joint 
Commission  on  Accreditation  of  Healthcare  Organizations  (JCAHO) 
core  measures. 

•  The  development  of  a  tool  to  provide  osteopathic  residency 
programs — and,  eventually  practicing  physicians — with  a  method  of 
measuring  the  quality  and  safety  of  care  they  provide  to  their  patients 
using  evidence-based  standards.  These  tools  are  organized  around 
clinical  conditions,  such  as  diabetes  mellitus  or  hypertension,  as  sets  of 
measures  pertinent  to  the  conditions. 

•  The  incorporation  of  the  process  of  continuous  quality  improvement 
into  graduate  medical  education  and  the  practice  of  osteopathic 
medicine. 

•  The  inclusion,  in  pertinent  measure  sets,  of  the  frequency  of  a 
documented  osteopathic  musculoskelatal  exam. 

Increasing  participation 

The  AOA  has  facilitated  the  development  of  the  AOA-CAP  to  evaluate  safety 
through  a  pilot  project  as  well  as  investments  in  infonnation  technology, 
epidemiology  and  quality  improvement.  The  CAP  has  been  developed  within 
family  practice  and  internal  medicine  residency  programs  and  will  be  offered  to 
practicing  primary  care  physicians  in  2005. 

The  development  of  the  measure  sets  has  been  guided  by  a  steering  committee 
composed  of  the  following  groups: 

•  American  Osteopathic  Association  (AOA) 

•  American  College  of  Osteopathic  Family  Physicians  (ACOFP) 

•  American  Academy  of  Osteopathy  (AAO) 

•  American  College  of  Osteopathic  Internists  (ACOI) 

The  steering  committee  is  responsible  for  assuring  the  measure  sets  are 
consistent  with  current  evidence-based  care,  and  is  also  responsible  for  oversight 
of  indicator  development  and  report  format. 
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After  incorporating  all  recommendations  into  the  measure  sets,  a  pilot  project 
of  the  AOA-CAP  was  initiated  in  1999.  Four  clinical  conditions — diabetes, 
women’s  health  screening,  childhood  immunizations,  and  osteopathic 
examination  of  the  spine — were  evaluated,  encompassing  more  than  30 
indicators.  The  pilot  project  was  offered  to  all  143  family  practice  residency 
programs  accredited  by  the  American  College  of  Osteopathic  Family  Physicians 
(ACOFP).  Twenty-seven  programs  indicated  an  intention  to  participate;  however, 
the  final  number  participating  in  the  program  was  1 1 ,  representing  family  practice 
residency  programs  and  1,541  patients. 

The  pilot  project,  completed  in  2001,  noted  a  variation  in  performance 
between  residency  programs,  and  identified  opportunities  to  improve  care  in 
participating  family  practice  programs.  The  ability  of  AOA-CAP  to  collect 
consistent  data  from  multiple  clinical  programs  and  provide  information  regarding 
performance  back  to  those  programs  was  also  realized. 

Evaluation  of  the  results  of  the  pilot  project  demonstrated  that  performance, 
using  the  NCQA  HEDIS  as  a  guide,  was  similar  to  national  averages  in  the  four 
clinical  entities. 

The  AOA,  through  its  Bureau  of  Osteopathic  Clinical  Education  and  Research 
(BOCER),  collaborated  with  the  ACOFP  to  explore  methods  of  increasing 
participation  in  the  AOA-CAP  by  family  practice  residency  programs. 
Acknowledging  low  participation  and  recognizing  the  importance  of  teaching 
continuous  quality  improvement  to  residents,  the  ACOFP  Committee  on 
Evaluation  and  Education  recommended  mandatory  participation  in  the  AOA- 
CAP  program  as  a  part  of  the  accreditation  process  for  AOA-approved  family 
practice  residency  programs.  This  step  was  approved  in  July  2003.  The  ACOFP 
Committee  on  Evaluation  and  Education  also  required  the  selection  of  one  of  two 
clinical  AOA-CAP  projects  during  2003-04  and  2004-05.  (The  clinical  areas  of 
focus  for  these  residency  programs  during  these  two  years  were  diabetes  mellitus 
and  low  back  pain.)  As  a  result  of  this  effort,  program  participation  in  the  first  two 
of  three  data  collection  periods  during  academic  year  2003-04  increased  from  the 
1 1  programs  in  the  pilot  project  to  more  than  40. 


Evolving  core  competencies 
in  the  practice  of  medicine 

As  the  IOM  and  others  have  begun  to  identify  deficiencies  in  the  current 
delivery  of  health  care,  a  model  of  core  competencies  is  beginning  to  emerge  from 
employer  groups  and  accreditation  agencies.  This  model  is  best  demonstrated  in 
the  Bridges  to  Excellence  initiative  supported  by  the  NCQA,  General  Electric, 
and  other  employers.6  The  model  includes  consistent  use  of  evidenced-based 
medicine,  understanding  the  concept  of  population  or  disease  management  using 
systematic  methods  of  improving  care  within  a  specific  clinical  entity,  and 
providing  a  continuum  of  care  for  patients  with  chronic  disease. 
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The  AOA-CAP  provides  residency  programs  with  the  framework  for 
development  of  these  competencies.  It  achieves  this  by  serving  as  a  method  of 
measuring  the  residency  programs’  compliance  with  evidence-based  standards  of 
care  in  the  management  of  patients  with  chronic  diseases  such  as  hypertension, 
diabetes  mellitus,  osteoporosis,  and  coronary  artery  disease.  The  CAP  also 
provides  information  on  frequency  of  preventive  interventions  including 
immunizations  and  cancer  screening. 

How  the  AOA-CAP  works 

With  the  engagement  of  all  family  practice  residency  programs  in  the  current 
AOA-CAP  brought  about  by  mandatory  participation,  the  number  of  clinical 
entities  covered  was  expanded  to  seven  clinical  conditions  (with  more  than  38 
indicators):  diabetes  mellitus,  coronary  artery  disease,  women’s  health,  low  back 
pain,  adult  immunizations,  hypertension,  and  childhood  immunizations. 

Presently,  the  program  is  a  retrospective  evaluation  of  care  delivery  within 
each  of  these  measure  sets,  which  has  a  well-developed  data  dictionary  consistent 
with  those  in  use  by  both  the  NCQA  HEDIS  initiative  and  the  JCAHO  Core 
Measure  initiative.7  Sample  size  for  each  clinical  entity  has  been  defined  by  using 
either  population  estimates  or  pre-and  postintervention  sample  frames.  Each 
participating  site  is  provided  standardized  materials  to  use  for  institutional  review 
board  submission,  with  the  assumption  that  the  project  will  be  submitted  for  an 
expedited  review  as  a  quality  improvement  project.  Reports  are  generated  for  each 
program  using  indicator  constructs  similar  to  those  used  by  external  bodies, 
including  both  the  NCQA  and  the  JCAHO,  with  the  flexibility  to  generate 
indicators  as  determined  by  the  steering  committee.  All  143  programs  with  more 
than  900  residents  are  expected  to  participate  by  the  end  of  the  2005  academic 
year. 

Project  description 

AOA-CAP  is  designed  to  be  an  ongoing  program  in  clinical  quality 
improvement.  Residency  programs  have  the  ability  to  register  their  programs 
through  the  AOA-CAP  Web  site,  accessed  through  DO-Online,8  which  is  the 
primary  method  of  communication  between  residency  programs,  the  AOA,  and 
the  principal  investigators  for  AOA-CAP. 

In  AOA-CAP,  perfonnance  is  measured  by  retrospective  abstraction  of 
required  data  elements  from  patients’  medical  records  by  residents  associated  with 
the  residency  program  and  trained  as  abstractors.  Data  elements  include 
demographic  information  (e.g.,  patient’s  age,  gender,  etc.)  and  clinical 
information.  Clinical  indicators  selected  for  measurement  represent  evidence- 
based  clinical  practice  standards  derived  from  large  randomized  controlled 
clinical  trials,  single  controlled  observational  studies,  or  expert  consensus.  Subject 
to  the  specific  measurement  set,  abstraction  time  per  record  requires  10-20 
minutes.  Patient  confidentiality  is  assured,  as  no  patient-identifiable  data  are 
collected.  The  elements  of  data  collected  by  the  AOA-CAP  meet  the  Health 
Insurance  Portability  and  Accountability  Act  of  1996  (HIPAA)  privacy 
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regulations  for  “de-identification  of  protected  health  information”  set  forth  in  45 
CFR  Sec.  164.514(b)(2). 

Residents  are  provided  with  comprehensive  data  dictionaries  and  trained  in 
data  abstraction  using  an  online  tutorial.  Data  dictionaries  provide  abstractors 
with  specific  procedures  and  definitions  to  ensure  the  uniformity  and  reliability  of 
abstracted  data.  Each  data  dictionary  has  a  general  description  of  clinical 
conditions  in  the  measurement  set;  information  necessary  to  precisely  select  the 
medical  records  to  be  abstracted;  and  detailed  background,  definitions,  and 
instructions  regarding  each  data  element  to  be  abstracted.  Data  may  be  abstracted 
onto  paper  fonns  and  later  transferred  to  an  online  data-entry  tool,  or  abstracted 
directly  from  the  medical  record  to  the  online  data-entry  tool  for  transmission  to 
the  AOA  for  analysis. 

AOA-CAP  provides  each  participating  residency  program  a  near  real-time 
report  of  current  clinical  practices  (15-30  days  following  the  close  of  a  data 
collection  period).  The  perfonnance  score  for  each  clinical  indicator  will  establish 
a  benchmark  against  which  residencies  can  compare  progress  in  future  studies, 
compare  performance  to  national  standards  (e.g.,  NCQA’s  HEDIS  measures),  and 
compare  their  performance  to  other  residency  programs. 

The  reports  generated  by  the  AOA-CAP  will  help  to  readily  identify  clinical 
practices  that  need  improvement  in  quality  and  intervention,  provide  a  guide  to 
curriculum  modifications,  serve  as  an  adjunct  to  teaching  tools,  and  offer  a 
stimulus  to  faculty  and  residents  to  continuously  improve. 

Benefits  to  family  practice  residency  programs 

The  AOA-CAP  offers  many  benefits  to  physicians  in  training.  With  growing 
concerns  regarding  patient  safety,  specifically  regarding  errors  of  omission  in  the 
ambulatory  setting,  the  AOA-CAP  provides  a  measurement  set  to  help  focus  these 
residents  on  improving  patient  care  and  exposing  them  to  health  care  quality 
improvement  techniques,  which  traditional  medical  education  does  not  generally 
deliver.  By  using  the  AOA-CAP  to  identify  opportunities  to  improve  care  and 
then  developing  systematic  methods  of  capitalizing  on  these  opportunities, 
residents  can  develop  competency  in  population-based  medicine  and  the 
dynamics  of  disease  or  care  management  of  patient  groups  with  chronic  disease. 
Thus,  residents  will  become  familiar  with  evolving  market-based  approaches  to 
quality  improvement,  such  as  the  Bridges  to  Excellence  program.  This  program, 
which  is  implemented  in  several  cities,  rewards  physicians  who  demonstrate  good 
outcomes  in  diabetic  care.6  In  addition,  the  CAP  provides  an  opportunity  to  better 
understand  health  services  research,  including  aspects  such  as  the  function  of 
institutional  review  boards;  the  different  dimensions  of  health  care  perfonnance 
measurement,  including  structure,  process  and  outcomes;  and  the  tools  needed  to 
quantify  each  of  these  dimensions. 
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A  quality  improvement  tool 

The  AOA-CAP  is  a  tool  that  provides  a  consistent  method  of  collecting  data 
on  health  care  delivery  within  ACOFP-approved  residency  programs,  and  can  be 
used  to  identify  opportunity  gaps  in  care  delivery.  Experience  with  the  AOA-CAP 
pilot  project  demonstrated  a  great  deal  of  interest  from  participating  programs  in 
understanding  why  some  programs  had  better  performance.  The  AOA  sponsored 
a  series  of  teleconferences  following  the  pilot  project,  highlighting  programs 
identified  as  “best  practices”  in  each  clinical  category.  The  teleconferences 
allowed  the  programs  to  share  their  processes  of  care  around  specific  indicators. 
Remeasurement  in  two  programs  that  contributed  diabetes  mellitus  data  to  both 
the  AOA-CAP  pilot  project  and  one  of  the  first  two  data  collection  periods  during 
academic  year  2003-04  showed  significant  improvement  in  most  indicators  of 
care.  Pooled  data  from  both  programs  demonstrated  a  reduction  in  poorly 
controlled  diabetic  patients  having  a  HgbAlc  >  9.5  percent,  from  32.10  percent  to 
7.02  percent  (P  <  0.001);  an  increase  in  influenza  vaccinations,  from 
14.81  percent  to  36.84  percent  (P  =  0.003);  and  an  increase  in  use  of  angiotension 
converting  enzyme  inhibitors  in  eligible  diabetics  with  albuminuria,  from 
72.73  percent  to  100  percent  (P  <  0.001).  The  AOA-CAP  can  serve  patient  safety 
as  a  quality  improvement  tool  and  provide  residents  in  training  with  the 
framework  for  understanding  the  model  of  continuous  quality  improvement. 


Initial  findings 

Diabetes  care  in  programs  participating 
in  the  first  two  data  collection  periods 

Diabetes  was  selected  as  an  initial  condition  for  participating  residencies 
because  it  is  a  serious  chronic  disorder  affecting  6.1  percent  of  Americans  and  is  a 
leading  contributor  to  death  and  disability.  Diabetes  complications  serve  as  the 
leading  cause  of  new  cases  of  adult  blindness;  account  for  about  one -third  of  all 
cases  of  end-stage  renal  disease;  represent  the  leading  cause  of  nontraumatic 
amputations  (54,000  per  year);  and  serve  as  a  major  contributor  to  macrovascular 
complications  that  include  heart  attack,  stroke,  and  other  life-threatening 
diseases.9 

The  Diabetes  Control  and  Complications  Trial  (DCCT)  and  the  United 
Kingdom  Prospective  Diabetes  Study  (UKPDS)  were  randomized  clinical  trials 
that  have  facilitated  recent  changes  in  the  recommended  treatment  of  diabetes. 

The  DCCT  and  UKPDS,  along  with  other  studies,  have  demonstrated  that  tight 
control  of  glucose  levels,  blood  pressure,  and  cholesterol  levels  can  delay  or 
reduce  the  onset  of  microvascular  complications  of  diabetes.10  14  Based  on  the 
support  from  these  studies,  the  American  Diabetes  Association  (ADA)  has 
developed  aggressive  clinical  practice  recommendations  for  the  diagnosis  and 
treatment  of  diabetes.  These  recommendations,  which  are  updated  annually,  were 
used  in  generating  the  AOA-CAP  diabetes  mellitus  report. 
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The  list  below  covers  the  quality  indicators  of  diabetes  care  in  the  AOA-CAP 
that  are  endorsed  by  the  AOA-CAP  Steering  Committee: 

•  Glycosylated  hemoglobin  (HgbAlc)  testing  and  degree  of  control  of 
HgbAlc  levels. 

•  Dilated  retinal  examination  in  past  year  recommended  and  performed 
by  qualified  specialists. 

•  Microalbuminuria  screening  conducted,  and  if  present,  appropriate 
treatment  with  ACE  Inhibitors  or  angiotensin  receptor  blockers 
(ARBs). 

•  Lipid  levels  taken  in  past  year  and  LDL  level  recorded  and  controlled. 

•  Blood  pressure  levels  recorded  and  control  of  HTN  in  diabetic 
patients. 

•  Foot  examinations  in  eligible  diabetic  patients. 

•  Recommended  immunizations  received. 

•  Osteopathic  assessment  perfonned. 

Comparison  data 

The  diabetes  mellitus  report  displays  the  following:  data  from  the  participating 
program,  the  average  measure  performance  of  all  ACOFP  accredited  programs 
contributing  data  from  the  same  time  period,  and  two  sources  of  external 
comparative  data.  The  first  comparative  dataset  is  from  the  National  Health  and 
Nutrition  Examination  Survey  (NHANES  1999-2000),  which  provides  an 
estimate  of  diabetic  prevalence  and  control  on  a  national  basis.15  The  second 
comparative  database  is  NCQA’s  HEDIS,  a  survey  of  managed  care  organizations 
in  the  country.5  This  infonnation  is  provided  to  give  a  valid  reference  for  how  a 
program  is  performing  in  relation  to  external  benchmarks.  Goals  for  each  program 
are  set  with  the  “best  care  for  patients”  under  consideration. 


Results 

The  following  tables  provide  an  example  of  what  residency  programs  receive 
during  participation  in  the  AOA-CAP.  The  information  provided  here  includes  all 
programs  contributing  data  to  the  first  two  data  collection  periods  covering  care 
provided  in  calendar  year  2002.  The  fonnat  is  designed  to  provide  participating 
programs  information  on  their  program  performance  along  with  comparative  data. 
The  measures  provided  are  based  on  evidence-based  practice  guidelines  derived 
from  large,  randomized  clinical  trials,  scientifically  designed  observational 
studies,  or  expert  consensus. 

Table  1  displays  demographic  information  regarding  those  eligible  diabetic 
patients  having  their  medical  record  abstracted  during  the  study  period  along  with 
comparative  data  from  NHANES  and  HEDIS.  The  study  period  for  this  report 
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Table  1.  Patient  demographics;  data  collection  periods  1  and  2 


Program  Identifier 

ABC 

Study  period:  2002 

(Example  program) 

All  CAP  programs 

Cases  abstracted  (#) 

37 

1486 

Average  patient  age  (#) 

56.2 

56.8 

Patient  age  range 

24-86 

19-89 

Gender-male  (%) 

43.2% 

40.5% 

Diabetics  on  insulin  (%) 

24.3% 

27.0% 

Comparison  performance  data 

Study  period 

NHANES 

1999-2000 

Average  patient  age 

NHANES 

59.3 

Diabetics  on  insulin 

NHANES 

27.4% 

was  the  calendar  year  2002.  The  AOA-CAP  participating  programs  have 
demographics  fairly  similar  to  those  from  national  representative  samples,  as  the 
AOA-CAP  data  featured  a  mean  age  of  56.8  years  and  the  NHANES  data  featured 
a  mean  age  of  59.3  years.  Additionally,  the  percentages  of  diabetics  receiving 
insulin  are  almost  identical  in  the  two  studies. 

Table  2  displays  glycemic  control  for  a  representative  program,  listing  figures 
for  the  percentage  of  patients  having  good  glycemic  control  (HgbAlc  <  7 
percent),  the  percentage  having  glycemic  control  in  need  of  focused  treatment 
action  (HgbAlc  >  8  percent),  and  the  percentage  having  poor  glycemic  control 
(HgbAlc  >9.5  percent).  Normal  (6  percent  or  less)  or  near  nonnal  (7  percent  or 
less)  HgbAlc  levels  are  associated  with  reduced  vascular  complications 
associated  with  the  disease.  Levels  above  8  percent  are  considered  “poor  control,” 
and  levels  above  9.5  percent  are  considered  “uncontrolled.”  Comparison  of 
participating  programs  with  nationally  representative  samples  shows  tighter 
glycemic  control  at  those  programs  contributing  to  AOA-CAP,  with  trends  toward 
lower  percentages  of  patients  with  HgbAlc  greater  than  9.5  percent  and  toward 
higher  percentages  of  patients  with  HgbAlc  less  than  7.0  percent. 

Table  3  demonstrates  perfonnance  in  preventive  care;  documented  screenings, 
including  eye  and  foot  examinations;  and  immunizations  given  to  diabetic  patients 
in  all  CAP  programs  and  at  the  program  used  as  an  example.  The  eye  examination 
portion  of  this  report  provides  an  illustration  into  several  unique  aspects  of  the 
AOA-CAP  reports.  First,  by  inclusion  of  the  frequency  of  recommendation  of  eye 
examination,  the  patient-physician  modifiable  barriers  to  performance  can  begin 
to  be  separated.  In  addition,  the  AOA-CAP  has  the  ability  to  construct 
performance  measures  according  to  external  or  internal  dictates.  The  eye  exam 
measure  consistent  with  HEDIS  definitions  (as  represented  in  the  row  named 
“done  HEDIS”)  allows  accurate  comparison  to  the  NCQA’s  HEDIS  measure.  The 
eye  exam  on  all  patients  (represented  by  the  row  named  done  all)  was  constructed 
differently,  allowing  the  steering  committee  to  evaluate  all  patients  having 
diabetes  mellitus  different  aspects  of  preventive  care  delivery. 
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Measures  that  do  not  have  commonly  accepted  constructs,  such  as  the  annual 
comprehensive  foot  examination,  are  developed  from  consensus  guidelines,  such 


Table  2.  Glcosylated  hemoglobin  (HgbAlc)  testing  and  level  of  control,  data  collection 
periods  1  and  2 


Program  Identifier 

ABC 

Study  as  of  2002 

(Example  program) 

All  CAP  programs 

Mean 

LCL 

UCL 

Mean 

Pgm  range 

HgbAlc  testing  done  (%) 

94.6% 

87.3% 

100.0% 

91.5% 

75-100 

Glycemic  control 

HgbAlc  <7.0% 

48.6% 

32.0% 

65.1% 

41.5% 

0-67 

HgbAlc  >8.0% 

28.6% 

13.6% 

43.5% 

31.8% 

13-60 

HgbAlc  >9.5% 

20.0% 

6.7% 

33.3% 

14.9% 

0-40 

Comparison  performance  data 

HgbAlc  (done) 

HEDIS 

2002 

82.6% 

HgbAlc  >  9.5%  (control) 

HEDIS 

2002 

33.9% 

HgbAlc  <  7.0%  (control) 

NHANES 

1999-2000 

37.0% 

HgbAlc  >  8.0%  (control) 

NHANES 

1999-2000 

37.2% 

LCL  =  Lower  Control  Limit 
UCL  =  Upper  Control  Limit 


Table  3.  Preventive  care  and  evaluation,  data  collection  periods  1  and  2 


Program  Identifier 

ABC 

Study  period:  2002 

(Example  program) 

All  CAP  programs 

Mean 

LCL 

UCL 

Mean 

Program  range 

Eye  care 

Eye  exam  (recommended) 

62.2% 

46.5% 

77.8% 

55.7% 

3.6-100 

Eye  exam  (done  HEDIS) 

48.6% 

32.5% 

64.8% 

42.4% 

3.6-100 

Eye  exam  (done  all) 

40.5% 

24.7% 

56.4% 

36.1% 

3-100 

Foot  care 

Examination  (done) 

56.8% 

40.8% 

72.7% 

55.7% 

0-100 

Immunizations 

Pneumovax  (given) 

13.5% 

2.5% 

24.5% 

31.6% 

0-100 

Influenza  vax  (given) 

32.4% 

17.3% 

47.5% 

38.6% 

0-100 

Comparison 

Eye  exam  (done) 

HEDIS 

Yr  2002 

51.7% 

LCL  =  Lower  Control  Limit 
UCL  =  Upper  Control  Limit 
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as  those  of  the  ADA.  The  ADA  recommends  that  such  an  examination  include  an 
assessment  of  the  following:  protective  sensation,  foot  structure  and 
biomechanics,  vascular  status,  and  skin  integrity. 

Influenza  vaccine  is  recommended  for  persons  at  increased  risk  of 
complications  from  an  acute  respiratory  illness.  Diabetic  patients  fall  into  this 
category.  Similarly,  high-risk  patients  should  receive  periodic  Pneumococcal 
vaccination.  Table  3  also  displays  performance  rates  for  these  two  clinical 
indicators. 

Table  4  demonstrates  control  of  hypertension  in  hypertensive  diabetics  using 
the  INC  VII  standard  of  less  than  130/80  mm  Hg.16  This  standard  was  published 
during  the  year  following  data  collection  (2003).  Control  to  this  blood  pressure 
provides  a  stretch  goal  for  programs.  Data  from  the  NHANES  study  suggest  that 
only  35.8  percent  of  all  diabetics  had  blood  pressures  in  the  recommended  range. 


Table  4.  Control  of  hypertension,  data  collection  periods  1  and  2 


Program  Identifier 

ABC 

Study  period:  2002 

(Example  program) 

All  CAP  programs 

Mean 

LCL 

UCL 

Mean 

Program  range 

Eye  care 

Diabetics  with  Hypertension  (HTN) 

70.3% 

55.5% 

85.0% 

71 .3% 

42-100 

Diabetics  with  controlled  HTN 

30.8% 

13.0% 

48.5% 

24.4% 

0-47 

Diabetics  with  controlled  HTN  on  ACEI 

85.7% 

70.1% 

100.0% 

87.0% 

53-100 

Comparison  performance  data 

Diabetics  with  Hypertension  (HTN) 

NHANES 

1999-2000 

51.4% 

LCL  =  Lower  Control  Limit 
UCL  =  Upper  Control  Limit 


Conclusions 

The  AOA-CAP  initiative  has  demonstrated  that  a  health  care  professional 
membership  organization  representing  practicing  osteopathic  physicians  can 
develop  and  implement  a  tool  to  measure  and  improve  ambulatory  patient  care 
safety.  The  AOA’s  high  level  of  commitment  to  improving  ambulatory  patient 
safety  and  confronting  criticisms  of  the  current  health  care  delivery  structure  has 
resulted  in  the  development  of  a  quality  improvement  program  geared  to  the 
graduate  medical  education  environment. 

The  implementation  of  the  AOA-CAP  has  been  enlightening  in  terms  of  the 
challenges  it  presented.  After  sunnounting  information-systems  and  methodologic 
issues  during  the  pilot  project  in  1999-2000,  the  steering  committee  was 
perplexed  by  the  low  level  of  engagement  with  family  practice  residency 
programs.  The  8-percent  enrollment  rate  of  family  practice  programs  during  the 
pilot  project  reflected  a  misunderstanding  of  registry  programs’  ability  to  improve 
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care  and  of  the  number  of  competing  interests  vying  for  residency  program 
directors’  attention.  In  an  effort  to  ensure  that  the  AOA-CAP  program  would 
receive  proper  commitment,  leadership  at  the  ACOFP  implemented  the 
mandatory  participation  requirement,  thus  assuring  that  ambulatory  patient  safety 
is  a  priority  for  osteopathic  family  practice  training  programs. 

The  two  family  practice  programs  that  participated  in  both  the  pilot  project 
and  one  of  the  first  two  data  collection  periods  of  the  current  project  have 
demonstrated  that  the  process  of  care  measured  in  diabetes  mellitus  is,  to  some 
extent,  modifiable  by  the  programs.  This  has  been  reinforced  by  findings  from  the 
NCQA  State  of  Managed  Care  report  that  has  demonstrated  improvement  in  the 
process  measures  tracked  by  HEDIS.5  Although  the  findings  may  be  biased  by  the 
fact  that  the  programs  were  volunteers  in  the  pilot  project,  it  is  clear  that 
improvement  of  measured  process  of  care  indicators  occurred. 

On  the  whole,  results  from  the  programs  participating  in  the  first  two  data 
collection  periods  of  the  current  AOA-CAP  program  are  fairly  similar  to  those 
found  in  national  studies,  including  NCQA,  which  is  representative  of  patients 
served  by  managed  care  organizations,  and  NHANES,  which  is  a  nationally 
representative  study  of  diabetic  patients.  Although  there  is  no  audit  function  to  the 
AOA-CAP,  the  similarities  of  the  findings  to  national  studies  suggest  that  the 
AOA-CAP  data  are  both  reliable  and  valid. 

Future  directions 

As  the  AOA-CAP  continues  to  be  refined  and  as  more  programs  become 
engaged  in  the  patient  safety  improvement  loop,  future  directions  of  the  program 
are  being  developed.  Areas  of  future  attention  will  include: 

Sharing  processes  of  care  between  residency  programs.  The  variation  in 
processes  of  care  seen  at  participating  family  practice  programs  provides  an 
opportunity  to  examine  the  differences  in  operations  between  programs.  Those 
operational  differences  that  result  in  higher  perfonnance  can  be  labeled  “best 
practices”  and  shared  with  other  family  practice  programs  in  an  effort  to 
accelerate  improvement.  This  sharing  follows  the  Institute  for  Healthcare 
Improvement  model  that  has  proven  successful  in  implementing  institutional 
change  in  health  care  providers.17  Models  of  sharing  can  include  highlighting 
“best  practice”  programs  at  regional  or  national  meetings  and  sponsoring 
teleconferences  that  provide  an  opportunity  to  share  results  and  practice. 

Use  of  the  registry  for  health  services  research.  The  registry  provides  a 

window  into  delivery  of  care  at  participating  programs.  Physician  or  patient 

factors  can  drive  variance  in  performance.  To  detennine  the  most  effective 

method  of  improving  care  it  is  necessary  to  understand  where  the  opportunity  gap 

exists.  The  model  for  residents  using  the  registry  to  answer  these  health-services 

research  questions  has  been  demonstrated  through  a  publication  from  the  pilot 

.18. 

project  that  evaluated  barriers  to  adequate  control  of  hypertension.  There  also  is 
an  opportunity  to  use  the  network  evolving  around  the  AOA-CAP  as  an 
osteopathic  primary  care  research  network. 
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Expansion  of  the  program  to  practicing  physicians.  With  increasing 
demands  to  demonstrate  performance  and  evolving  pay-for-performance 
initiatives,  such  as  Bridges  to  Excellence,  practicing  physicians  may  find  the 
AOA-CAP  an  important  tool  to  fulfill  such  needs.  The  program  could  be  changed 
to  fit  the  requirements  of  external  users,  such  as  the  Centers  of  Medicaid  and 
Medicare  Services  or  other  payers.  The  AOA-CAP  is  one  method  of  addressing 
the  difficult  questions  raised  by  the  various  IOM  reports  regarding  patient  safety 
and  improving  quality  care. 
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